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Abstract. We define a large class of quantum sources and prove a quantum analog of the 
asymptotic equipartition property. Our proof relies on using local measurements on the 
quantum source to obtain an associated classical source. The classical source provides an 
upper bound for the dimension of the relevant subspace of the quantum source, via the 
Shannon-McMillan noiseless coding theorem. Along the way we derive a bound for the von 
Neumann entropy of the quantum source in terms of the Shannon entropy of the classical 
source, and we provide a definition of ergodicity of the quantum source. Several explicit 
models of quantum sources are also presented. 
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I. Introduction 

I. A. The possibility of building a quantum computer has stimulated new interest in the 
quantum analog of classical information theory (see [Be] for an introduction and review of 
current ideas). Shannon, McMillan, Khinchin and others provided a firm foundation for 
the classical theory, and used the mathematics of stochastic processes to prove important 
theorems. In particular they obtained limits on the amount of information that can be 
transmitted through a channel. Although a quantum computer does not yet exist, it is 
reasonable to suppose that similar issues of channel capacity are relevant to its operation. 
So it is interesting to investigate this question, using our current understanding of quantum 
mechanics. 

There has been much work done on this and related questions. In particular, Schu- 
macher stated and proved a capacity result for a quantum channel in [S], [JS]. Our partic- 
ular interest is in the extended quantum signal source described by Schumacher. This is a 
quantum system whose state space is a (tensor) product of many copies of one fundamen- 
tal state space M. The source produces a signal which is encoded by a state in M; the 
ensemble of possible signals is represented by a density operator p on M. The extended 
source corresponds to a sequence of such states, and has a natural interpretation as a mes- 
sage. The probabilistic character of the message is contained in the density operator on 
the tensor product of copies of M. One choice of density operator is p®. . .®p. This corre- 
sponds to independent signals at all times, and there are no correlations between signals in 
the message. For this reason we call this a quantum Bernoulli source (a precise definition 
is provided in section II). Schumacher proves his quantum noiseless coding theorem for 
such a source. He introduces the notion of fidelity of a quantum channel, and states his 
results in terms of this. On the most basic level his results show that the state space for 
the extended source has a relevant subspace whose dimension is determined by the von 
Neumann entropy of p. As far as the information content of the source is concerned, the 
rest of the state space can be ignored. 

I.B. Our main interest in this paper is to extend Schumacher's result to a large class 
of quantum sources. We define precisely what we mean by this in section II. Roughly 
speaking, we consider sources which allow correlations on all time scales between signals in 
a message. In classical information theory the coresponding result is the Shannon-McMillan 
theorem, which shows that under very general assumptions the ensemble of all possible 
messages can be split into a relevant and an irrelevant part. The criterion for splitting is 
provided by the entropy of the source. The corresponding result for the quantum theory 
should be a splitting of the state space into a relevant subspace and an irrelevant subspace, 
with the von Neumann entropy as the criterion. This is precisely what Schumacher proved 
for the quantum Bernoulli source. We obtain such a splitting in the general case, and 
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derive an estimate for the dimension of the relevant subspace, by computing the entropy 
of a classical source which is obtained by making measurements on the quantum system. 

To summarize, we show that a general quantum source can be encoded by a quantum 
system whose dimension is smaller than the original. We give estimates for the dimen- 
sion of the reduced space, based on the results of local measurements of the source. For 
the case of a quantum source emitting orthogonal states our estimate is tight, and for a 
Bernoulli source reproduces Schumacher's result. We also derive an inequality relating this 
experimental entropy to the true von Neumann entropy. 

I. C. The paper is organized as follows. In section II we define quantum sources, and 
recall some standard results about the construction of infinite quantum systems. We 
also provide some explicit examples which serve to illustrate our ideas. In section III we 
recall the classical notion of an ergodic source, and propose a definition of ergodicity for 
a quantum source. As a check we prove that a quantum Bernoulli source is ergodic. In 
section IV we recall the notion of quantum entropy. In section V we pursue the quantum 
analog of the Shannon-McMillan theorem. To do this we use the notion of a positive 
operator valued measure, and construct an associated classical source. The entropy of 
this associated source satisfies a lower bound involving the von Neumann entropy of the 
quantum source. It also determines the dimension of a relevant subspace of the quantum 
system which in turn yields our quantum noiseless coding theorem. 

II. Quantum sources 

II. A. According to the mathematical theory of information, see e.g. [A], [B], [K], a (clas- 
sical) source is a stochastic process. In this paper, we are concerned with discrete sources 
which can be described as follows. We are given a finite set X, called the alphabet, and 
consider the space X°° of all infinite sequences x = {x n } ne z, called messages. The time 
evolution is given by the shift T : X°° — > X°° defined by (Tx) n := x n+ i. The stochas- 
tic character of the process is governed by a probability measure \i on X°° . The triple 
(X°°,T, (i) is called a source. We say that the source is stationary, if T preserves the 
measure fx. 

We begin by describing informally the properties of a quantum source. In the next 
section we will provide a rigorous mathematical description. By analogy with the classical 
case, a quantum source will be a triple consisting of quantum messages, the time shift, and 
a probability distribution for the messages. For technical reasons, it is useful to describe 
the space of quantum messages (which is a linear space) by the algebra of observables on 
it. 
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A quantum source sends a series of signals, each of which is a vector in a finite di- 
mensional Hilbert space H. We assume that the source is discrete, i.e. each signal is an 
element of a finite set S = {\ipi), ■ ■ ■ , \ip s )} of normalized vectors in H. To avoid unnec- 
essary redundancy, we assume that the space H is spanned by 5, i.e. 7i ~ C d , where 
d < s. We do not assume that the elements of S are orthogonal to each other or even 
linearly independent. Indeed, this is an important difference from the classical situation 
which does not allow for forming linear superpositions of states. Denote by pj the a priori 
probability of the state being sent. The density matrix corresponding to the ensemble 
of signals S is then given by 

p= E PjMte/.- (n.i) 

l<j< s 

As a consequence of our assumptions, tr(p) = 1. Clearly, an ensemble of signals S and the 
associated a priori probabilities determines uniquely the density matrix p. On the other 
hand, any given density matrix corresponds to infinitely many different sets of signals. For 
a discussion of this point, see [HJW]. 

The observables associated with quantum signals are d x d hermitian matrices; more 
formally (we will need this viewpoint shortly) they are elements of the C*-algebra A = 
C(H) of linear operators on H. Abusing slightly the language, we will refer to A as the 
algebra of observables. Using the language adopted in the operator algebra approach to 
quantum physics, see e.g. [H], [BR], we define the state on the algebra of observables A 
associated with the density matrix p to be 

n(A):=tr(Ap)= £ Pj (^\A\^)- ("-2) 

l<j<s 



II. B. We now propose a formal definition of a quantum source. To this end, we construct 
the state on the algebra of observables associated with entire (infinite) quantum messages 
rather than individual signals. This is technically somewhat delicate, as it involves infinite 
tensor products of Hilbert spaces. Let I C Z be a finite set of the form {M, M+ 1, . . . , N — 
1 , A}, where M < N, i.e. / is a finite collection of consecutive integers. By X we denote 
the partially ordered set of all such J's. We set S)i := ^^jHj, where Hj = H for 
all j G /, and define the corresponding observable algebra 21/ := C(S)i) ~ C(H), 
where |/| denotes the number of elements in /. For Jc J, there is a natural embedding 
£($)i) ^ C{S)j), and so we can form the union 2l; oc := |J JeX 2l/. The latter is a normed 
algebra, and we refer to its elements as local observables. Roughly, 2lj oc is a collection of 
operators acting on the infinite tensor product j6Z 7ij; every element of 2l/ oc acts as 
the identity on all but a finite number of factors in this product. For a local observable 
A E %oc, we let supp(A) denote its support, i.e the smallest I el such that A e 21/. The 
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norm closure 21 of 2l; oc is a C*-algebra called the algebra of quasilocal observables. These 
concepts are borrowed from algebraic field theory and statistical mechanics, and we refer 
the reader to [H] and [BR] for a thorough presentation. 

We will use the net {2l/}/ e x of matrix algebras to construct the Hilbert space of 
states of a quantum source. Assume that we have a family {Hj}j e j, 11/ e 21/, satisfying 
the following assumptions: 

1°. Each 11/ is a positive operator. 

2°. If | J | = 1, then 11/ = p. 

3°. Let I C J be such that J\I G J. Then 

trs, Uj = U AI , (113) 

where tr^ denotes the partial trace over the factor S)i in the tensor product Sjj = 

ft i ® or #j = £ j\i <8> 
Note that the last condition implies, in particular, that tr IT/ = 1. In other words, {Hi}i e j 
is a consistent family of density matrices, and it can be thought of as a quantum mechanical 
counterpart of a consistent family of cylinder measures. 

For each I £ I we define a state tj on 21/ by 

n(A) :=tr(AII/), (II.4) 

and observe that |tj(A)| < \\A\\, uniformly in I. The consistency condition (II. 3) implies 
that Tj is well defined, and that the generalized limit t(A) := lini/ tj(A) exists for all 
A E 21/oc, and satisfies |r(A) | < \\A\\. As a consequence, r can be uniquely extended to a 
state on the C*-algebra 21 of quasilocal observables. We use the same symbol r to denote 
this extension. 

Let fi,7r be the GNS representation, see e.g. [BR], associated with the state r. The 
Hilbert space ft is the state space of the quantum source. If no confusion arises, we will 
write A instead of tt(A). 

The additive group Z underlying the above construction plays the role of (discrete) 
time translations. Its action on the algebra of local observables is defined as follows: 

21/ 3 A~ A® I — ► a(A) :=I(g)A~Ae & J+1 . (II.5) 

In other words, a pushes the observable to the right by one unit of time. Clearly, ||ct(A) || = 
||A||, and so a has a unique extension to all of 21 which we will denote by the same symbol. 
The family of automorphisms {a n } ne z defines then a representation of Z. The triple 
(21, a, r) is called a quantum source. 

We say that the quantum source is stationary, if the state r is invariant under a, i.e. 

r(a(A)) = r(A), (II.6) 
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for all A G 51. From now on we will be assuming that our source is stationary. A standard 
result in operator algebras implies that the automorphism a is unitarily implementable on 
the GNS Hilbert space associated with the invariant state r, i.e. a(A) = FAF~ X on fj. 
We will call the unitary operator F a quantum shift. 

Because of stationarity, we can always assume that / G X is of the form {1, . . . , n}. 
To simplify the notation, we will write then Yl n instead of Ilj, and r n instead of 77. 

II. C. The simplest example of a quantum source is a Bernoulli source, which we now 
describe. As in the classical case, a Bernoulli source produces messages which are sequences 
of independent signals. Accordingly the family of density matrices {IT/} is given as follows: 

n 7 = ® ieI p h (11.7) 

where each pi equals p. It immediately follows that the consistency condition (II. 3) is 
satisfied, and that the source is stationary. This is the class of sources considered by 
Schumacher [S]. One special feature of a Bernoulli source is that whenever supp(A) n 
supp(S) = 0, we have t(AB) = t(A)t{B). 

There are many examples of non-Bernoulli sources. We present here a special class 
of stationary sources. These are all described by a signal density matrix p and another 
matrix R G C(H®H) satisfying 

tri((p <g> I)R) = p, tr 2 (R) = I. (II.8) 

Here / is the identity matrix on H, and we introduce the notation tr^ for the partial trace 
over the ith factor in the n-fold tensor product ® n 7i. The density matrices {II n } are then 
constructed recursively as follows: 

ni = p, 

1 1 
n n+1 = -(n n ®i 1 )(i n _ 1 ®R) + -(i n _ 1 ®R)(n n ®i 1 ). 

We have denoted by I n the identity matrix on the product ® n 7Y. The consistency of this 
definition ((II.3)) is immediate. The positivity of the density matrices II n is a further 
constraint on R. We have several explicit examples for which the positivity can be proven. 
In the simplest situation the matrix R satisfies the following additional conditions: 

R > 0, 

[p®h,R] = 0, (II.9) 
[h ®R,R®h] = 0. 
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It follows readily that all the matrices II n are positive, for all n > 1. For example, sup- 
pose p = X^Li AjPj, where {Xj} are the eigenvalues of p, and {Pj} are the corresponding 
orthogonal projections. Then we can take R = Y?j=i Pj ® Pj> an< ^ au the above properties 
are easily seen to hold. 

For our second example H = C 2 , and we assume that p is strictly positive. Let {<jj} 
be the Pauli matrices: 

ai = (l J)' a2= (z 0*)' a3= (o -l)- 
By choosing a suitable basis we can write 

P=\l+^ (11.10) 

where \a\ < 1. Then we take the matrix R to be 

a 1 

R = I®p+(-I- -a 3 ) ® (6(7i + ca 2 ). (11.11) 

— ■ ^ 

The consistency conditions are easily verified. Positivity of all matrices {IT n } holds for 
\b\, \c\ sufficiently small; the proof is given in the Appendix. 

II. D. The density matrix II n can also be written in terms of the states {ipj} which span 
S (compare (II. 1)) as follows: 

Hn= Yl Pjl,-,JnlV'il)<^ll®---®IV'i„)<V'in|- 

l<Jl,---,Jn<S 

The numbers Pj 1 ,...,j n satisfy obvious consistency conditions. If the states {ipj} are linearly 
independent, then they are also non negative. Further, if the states are orthogonal, the 
density matrices {n n } commute. In this case, the quantum source is equivalent to a 
classical source. 

III. Ergodic quantum sources 

III. A. According to the individual ergodic theorem [CFS], for any function / e L 1 (A'°°), 
the sequence of time averages 

</>*(*):= E ( IIL1 ) 

0<n<N-l 
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converges almost everywhere to a limit /. The function / is invariant under T. The 
source (X°°,T, fx) is called ergodic, if the only functions invariant under T are constants. 
Consequently, for an ergodic source / = f Xoo f(x)d/j,(x). This condition is one of the 
several equivalent statements which could be used to define ergodicity, see e.g. [CFS]. 
There is no natural concept of convergence almost everywhere in quantum mechanics. 
Consequently we adopt a notion of convergence which is natural for the algebra 21. Our 
definition of ergodicity of a quantum source is specifically designed to suit the purposes of 
this paper, although it may have broader applications. 

We say that a quantum source is ergodic if the following condition is satisfied. For 
any A G 21, the time averages 

(A) N : = £ F n AF~ n (III.2) 

0<n<iV-l 

converge in a suitable sense to the limit r{A)I, as N — > oo. Specifically, we require weak 
convergence, i.e. for all 0, ip G ,£), 

lim (0,((^) A r-T(^)/)^)=O. (III.3) 

N—>oo 

For A G 21, we let [A] denote the corresponding element of Substituting = [I] and 
= [B], Be Qlioc in (HI. 3), we find that for an ergodic source, 

lim t((A) n B) = t{A)t{B), (III.4) 

N^oo 

for all A,B G 2l; oc . 

III.B. The simplest example of an ergodic source is a Bernoulli source. 

Proposition III.l. A quantum Bernoulli source is ergodic. 
Proof. Let A,B,C G 2lz oc . We assert that 

t(B^(A) n C) - r(B^C)r(A), (III.5) 

which is equivalent to (III. 3) with <p = [B], ip = [C]. To prove this, we observe that there 
is no such that 

T{B^F n AF- n C) = r(F n AF~ n )r(B^ C) = t(A)t(B^C), 

for n > n (this follows from the fact that the supports of A, B, and C are all finite.) 
Consequently, 

T{BUA) N C) = ± t(B^ F n AF~ n C) + N ~l°~ 1 t{A)t{B^C). 

0<n<no 
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But 

1| r(B^F-AF--C)\<^±^\\A\\\\B\\\\C\\^0, 

0<n<n 

and so limjv^oo r(B^(A) N C) = t(A)t(B^C). 

The remainder of the proof is a series of straightforward approximation arguments. 
1°. We claim that (III. 5) holds for all A,B,C<E 21. To prove this, observe that for all N, 

\\{A) N \\ < \\A\\. (111.6) 

For A e 21, let Aj e % oc be such that \\A - AA\ -> 0, as j -> oo. For any B,C e % oc , 
write 

r(st(A) 7V C - r(A)stc) = r(B^(A j ) N C - T(Aj)B^C) 

+ r(B^{A-A j ) N C)+r(A j -A)r(B^C), 

and choose j such that \\A — Aj\\ < e/(3||S|| ||C||). Now choose Nq = No(j) such that 
\T(B^(Aj) N C - r(A J )stc)| < e/3, for all N > N . Then, using (III.6), 

\r(B j f(A) N C-r(A)B j fC)\ < e/3 + || (A - Aj) N \\ \\B\\ \\C\\ + \\A 3 - A\\\\B\\\\C\\ 

< 2\\A - Aj\\\\B\\ +e/3 < e, 

for all N > N Q . We have thus shown that (III.5) holds for all A e 21 and B,C E 
2l; oc . Repeating twice almost verbatim the above 3e argument we establish (III. 5) for 
all A,B,Ce 21. 

2°. Having established (III. 3) for (p = [B], ip = [C], and A e 21, we now show that it holds 
for arbitrary 0, ip e ft. Let (/> e ft, and let Sj be such that \\<f> - [Bj]\\ < e/(2|| [C] || || 
Write 

(0, ((^ - r{A)I)[C\) = ([ Bj ], ((A) N - t(A)I)[C}) + (cf> - [Bj], ((A) N - r(A)I)[C}), 

and choose N = N (j) so that | ([Bj], ({A) N - r(A)I)[C})\ < e/2, for all N > N . Then 

| (0, ((A) at — t(A)I) [C]) I < e/2 + ||0 - [5,-] || || [C] \\ \\A\\ < e. 

Repeating this argument we establish (III. 3) for all ip G ft. The proof of the proposition 
is complete. 
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IV. Entropy of a quantum source 

IV. A. In this subsection we construct the entropy of a quantum source. Our construction 
is largely standard, see e.g. [LR] and [W] , but we include most of the details to make the 
presentation self contained. The key mathematical input is the following lemma whose 
proof can be found on page 1122 of [LR]. 

Lemma IV. 1. Let A and B be positive trace class operators on a Hilbert space. Then 

tr(AlogA- AlogB) > tr(A-B). (IV.l) 

We define the entropy associated with a sequence of n signals to be 

H n (IL) := -tr w ®„(n n lo g n n ). (IV.2) 

Substituting A = Ii m+n and B = II m <g)II n in (IV.l), we obtain the following subadditivity 
property of H n (U): 

H m+n (U) < H m (U) + H n (U). (IV.3) 
A standard argument, see e.g. [K], pages 48-49, shows that the limit 

h(U) = lim - H n (U) (IV.4) 

n^oo fl 

exists. We call h(H) the entropy of the quantum source. Notice also that subadditivity 
implies the inequality 

h(U) < -tr w (plogp) < logd, (IV.5) 

where p is the density matrix for the signals, and d is the dimension of the signal Hilbert 
space H. 

IV. B. We can easily compute the entropy of two basic types of quantum sources introduced 
in II. 

For a Bernoulli source, II n = p <g) . . . <S> p, and so 

- H n (U) = -- tr n ® n ((®» (log®») 
n n 

= ^ tr w ®« ((® n /9) (J(g)...(g)logp(g)...(g)/)) 

l<j<n 

= -tr w (plogp). 

As a result, h(U) = — tr(plogp), i.e. the entropy of a Bernoulli source is equal to the von 
Neumann entropy of the signal density matrix. 

For the non-Bernoulli source defined in (II. 9) a similar calculation yields h(H) = 
— tr((p<g) I)RlogR). We have not found a closed form expression for the entropy of the 
other non-Bernoulli source described in II. 
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V. Asymptotic equipartition property 

V.A. The classical asymptotic equipartition property, also known as the Shannon- 
McMillan theorem, see e.g. [A], [K], states that if (X°°,T, /u,) is an ergodic source with 
entropy h(n), then the sequence 

f n (x) := -- log fi({y G X°° : y x = x u . . . , y n = x n }) (V.l) 

converges in measure to h(y). In other words, given 5, e > 0, there is uq such that for all 

n > no, 

V({x E X : \f n {x) - h(fj)\ > 6}) < e. (V.2) 

This is interpreted as saying that for any length n, there are two categories of messages 
sent by a source: (i) a small fraction of "likely" messages, each of which carries equal 
probability 

fi({yeX°°: Vl = x u ...,y n = x n })~e- nh ^\ (V.3) 

and (u) the bulk of "unlikely" messages whose total probability goes to zero as n goes to 
infinity. There are approximately e nh ^ likely messages which is much less than the total 
number of messages e nlogr (unless h{pt) happens to equal logr). 

The goal of this section is to establish an analogous result for quantum sources. 
Our theorem generalizes Schumacher's result [S], [JS] to general, not necessary Bernoulli, 
sources. 

V.B. Let A = {Ai, . . . , A r }, r < oo be a family of observables on TC such that Aj > 0, for 
all j, and 

A 1 + ... + A r = I. (V.4) 

Such a family is called a positive operator valued measure (POM), see [HJW] and references 
therein. A POM is called pure if each Aj is a rank one operator. For example, any family 
of d = dimH pairwise orthogonal projections on H satisfies the above conditions and so is 
a pure POM. We will call the set Xa = {1, . . . , r} the classical alphabet associated with 
the POM A, and denote by X^ the space of all infinite messages over the alphabet Xa- 
We can define a probability measure on X^ associated with the quantum source (21, r, a). 
For {ki, . . . , k n } E X n we define 

A»n ({x '■ x 1 = k 1 ,...,x n = k n }) := r n (A kl ®...®A kn ). (V.5) 

This defines a consistent family of cylinder measures, as 

^] Hn+idx '■ xi = k\, . . . , x n = k n , x n +i = /c n _|_i}) 

= r n+1 (A kl ® . . . ® A kn A k) 

= r n+1 (A kl ® . . . (8) A kn (g) /) = r n (A kl ® . . . <g> A kn ) 
= A*^({2i : Xi = ki, . . . , x n = k n }). 
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Let fi A denote the probability measure obtained from {/U A } by means of Kolmogorov's 
extension theorem. By T we denote the shift operator on X^. Then the triple T, /j A ) 
forms a classical information source. We emphasize that it depends on the choice of A. 
Obviously, this source is stationary. 

V.C. In fact, this source is ergodic if the underlying quantum source is ergodic. We state 
it as the following lemma. 

Lemma V.l. If (21, a, r) is ergodic, then for any choice of A, the classical source con- 
structed above is ergodic. 

Proof. We show that (Xjf, T, / u A ) has the following property: for every / G L l (X°°, dfi A ), 
the sequence (/)at converges in L 1 to J f dfi . Using Fatou's lemma, and recalling the 
individual ergodic theorem, this implies that the only functions invariant under T are 
constants, and so the classical source is ergodic. We first observe that it is sufficient to 
show that if C is a cylinder set, and xc denotes the corresponding characteristic function, 
then 

/ \(xc)N(x)-^ A (C)\d^ A (x)^0, (V.6) 

as N — > oo. A standard "e/3" argument then implies that for all / G L 1 (A'°°, d/u, A ), (/)n 
converges to f f d^ A in the L 1 -norm. Furthermore, since d/i A is a probability measure, 
(V.6) will follow from convergence of (xc)n to i~i A (C) in the L 2 -norm. This in turn is 
implied by the following stronger result. Suppose C and V are cylinder sets, and xc an d 
Xt> are the corresponding characteristic functions; then 

/ (xc)n{x) X v{x) d^ A (x) — ^ A (C)^ A (P), (V.7) 

as N — > oo. 

In order to prove (V.7), let C = {x : x\ = k\, . . . , x n = k n }, V = {x : Xj+i = 
li, . . . , Xj +m = l m }. The corresponding observables are given by 

G(C) = A kl <g> . . . <g> A kn , G(V) =A ll ®...®A lm . 

There is N such that for all n>4 T"(C)nD = |. Therefore, 



N-l 

.A, 



^ / E Xc(T*x) X v(x) d^(x) 

Jx a i=N + l 



= ^ T ( E «^(G(C))G(D)) 

i=iV + l 

1 N 

= t((G(C)) n )G(V)) - - t(J2*HG(C))G(V)). 



i=0 
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By our assumption of quantum ergodicity, the first term above converges to fi A (C) fi A (V) , 
as N — > oo. The second term is bounded as follows: 

1 N ° /\r _i_ i n 1 

-t(5><(g(c))g(2>)) <^nii^n niKii 5 



i=0 p=l q=l 

which converges to as N — > oo. Finally, 

„ / E^(^x) Xc (x) ^ A (x) < > 0, 

and the proof is complete. 

V.D. Let tiA denote the Shannon entropy of the classical source {X A , T, / u A ) ([B], [K]). 
In other words, 

h A = ~ lim - /«a({^: ffi = fci, . . . ,x n = fc n }) 

fci,...,fc„eAf A 

xlog/UA(fe: zi = fci, . . . ,x n = k n }) 

1 

= - lim - V r(A kl (8) ... (8) A fe J logr(A fcl <g> . . . <g> A fe J. 

n^oo 77, • 

ki,...,k n eXA 

A remarkable fact about ft a is that it can be bounded from below in terms of ft(n), the 
quantum entropy of the source, and a quantity depending exclusively on the statistical 
properties of the signal ensemble. 

Theorem V.2. For any POM A the following inequality holds: 

h A >h(U)- tr(A fc p)logtr(A,-). (V.8) 

l<fc<r 

Remark. In particular, if A is a pure POM consisting of d mutually orthogonal projec- 
tions, then /7a > h(H). 

Proof. The proof of this theorem is based on the following lemma. 

Lemma V.3. Let T be a Hilbert space of finite dimension N, and let R be a density 
matrix on T . If Ai, . . . , A r are positive operators such that A\ + . . . + A r = I, then 

tr(A JJ R)logtr(A JJ R) < tr(i?logi?) + ^ tr(A,-.R) logtr(A,-). (V.9) 
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Proof. Let Aj l denote the matrix entries of Aj in an orthonormal basis consisting of 
eigenvectors of R. Then tr(AjR) = J2 k ^kAj h , where Ai, . . . , Ajv are the eigenvalues of R. 
The function [0, 1] 3 x — > f{x) := xlogx is convex, and so by Jensen's inequality, 

A k k A kk 

E/(E ^ A t) = E E tA-M + E E tA~^ 

j k j k ^ 3> j k ^ 3 > 

< E E A t + E tr ^) lo s tr (^) 

j k k 

= E^ A *)+E tr ^) lo s tr (^), 

fc A: 

which implies (V.9). 

As a consequence of this lemma, and condition 3° in the definition of a consistent 
family of density matrices, 

tr ((A kl (8) ... (8) A fc JII n ) logtr ((A fcl <g> . . . <g> A fc „n n )) 
< tr(n n lo g n n ) + tr((A fcl <g> ...<g> A fc jn n )logtr(A fcl <g> . . . <g> A fcn ) 

= tr(n n lo g n n )+ tr((A kl ®...®A kn )Il n )\ogtr(A k .) 

l<j<n fci,...,/c„6A'A 

= tr(n n lo g n n ) + n tr(A fc p)logtr(4 fc ), 

l<fc<r 

and the claim follows. 

V.E. The theorem below is the main result of this section. It can be regarded as a quantum 
version of the Shannon-McMillan theorem. 

Theorem V.4. Let (21, r, a) be an ergodic source, and let A be a POM, for which the 
operators {Aj}, 1 < j < r, are orthogonal projections. Let hx be the Shannon entropy of 
the associated classical source, and define 

M — max rank (A,), m = min rank (A,). 

l<j<r ' 

Then, given 5, e > 0, there is uq such that for all n > n , 

where S n is a subspace whose dimension satisfies 

\ogm-5 < log dim£ n _ h A < log M + 5 
logd ~ log dim 'H® n \ogd ~ logrf 
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Further, let Ps n be the orthogonal projection onto S n . Then for any observable C G 
C(H® n ), 

\r(CP Sn )-r(C)\<e\\C\\. (V.ll) 

Remark 1. We can regard S n as a significant subspace of r ri® n in the sense that the 
expectation of any observable is almost completely determined by its restriction to <S n . If 
the states {\ipj)} are orthogonal, we can take the d orthogonal projections {\ipj)(ipj\} for 
the POM. In this case the entropy ft a is equal to the von Neumann entropy, since the 
density operators all commute. Then there is a direct correspondence with the classical 
Shannon-McMillan theorem, and the quantum theory is just a restatement of the classical 
result. 

Remark 2. In the Bernoulli case we can take the POM A to be d orthogonal projections 
onto the eigenvectors of p, in which case the inequality of Theorem V.2 is saturated. If 
p has simple spectrum, this means that the Shannon entropy equals the von Neumann 
entropy of the quantum source. Our result then agrees with Schumacher's conclusion that 
the information contained in the quantum source resides in a subspace whose dimension is 
asymptotically e nflA . In the general case we obtain only an upper bound for the dimension 
of the relevant subspace, and this upper bound depends on the choice of POM. For example, 
if each operator Aj in the POM A is equal to (l/d)I where I is the identity, then the 
Shannon entropy is h a = logd. Since this is the maximum possible entropy for a POM 
with d operators, we can conclude that all information about the quantum source has been 
lost in this measurement process. As these results show, it is advantageous to use a POM 
composed of orthogonal projections. 

Proof. We use the POM to construct the classical source Xjf, with entropy h A . Let f n (x) 
be the empirical entropy of a message x of length n defined in (V.l). Given e, 5 > let us 
define the sets 

U n ,s = {x G X : \fn(x) - h A \ > 6}, 
L n ,8 = {x G X : \f n (x) - h A \ < 5}. 
By the Shannon-McMillan theorem, given e, 5 > 0, there is n such that for all n > n , 

^a({^}) < e. 

xeu n ^s 

Since the operators {Aj} are orthogonal projections, each tensor product A^ <S> ■ ■ ■ <S> A^ n 
is an orthogonal projection, and hence so is the sum of these operators over the set L n j. 
Let <S n denote the range of this projection, and let Ps n denote the orthogonal projection 
onto this subspace. Then for any observable C, we have 

\r(CP Sn )-r(C)\<e\\C\\. 
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It remains to estimate the dimension of «S n . Since the projections {Aj} are orthogonal, 
its dimension is given by 

n 

dim(S n ) = ^ JJrank(A Xj ). 

Therefore, 

m n \L n)S \ < dim(S n ) < M n \L n , s \, 
where |£ n ,<5| is the size of the set L n g. The Shannon-McMillan theorem implies that 

(1 - e )e n{hA+5) > \L n>s \ > e n{hA - S) . 

This leads to 

logm 5 dim(S n ) hx < logM 5 log(l — e) 
logd \ogd ~ nlogd logd ~ logd logrf n 

and the result follows. 



Appendix 

We present here the proof that the i?-matrix (11.11) defines a family of positive density 
matrices, when \b\ and |c| are sufficiently small. It is convenient to introduce the matrices 

a 1 

Q = p~ 1 (bai + ca 2 ). 
Note that we assume \a\ < 1, so p~ x exists. Then for all n > 2 we define 

S n = I n - 2 ®u®Q. 

It follows by direct calculation that for all n > 1, 

n n+1 = n n ® P + ^(n n ® P )s n+1 + -^ +1 (n n <g> P ). 

In order to proceed we make the inductive assumption that II n > 0; this implies in par- 
ticular that (II n + u)~ l is bounded for every u > 0. We will prove that (n n+1 + ti) -1 is 
bounded for every u > 0; together with the positivity of 111 = p, this will establish the 
desired result. 
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Note first that \\S n \\ < \\uj\\ \\Q\\, and this bound is uniform in n. For convenience we 
define 

w=IM|, 9=IIQII- 

Furthermore, for any u > 0, 

(n n+ i + u)- 1 =(n n ® P + u)-\i + ^(n n ® p)s n+ i(n n ® P + u)- 1 
+ ^s£ +1 (n n ® P )(u n ® p + u)- 1 ) -1 . 

Our inductive assumption implies that (II n <g> p + -u) _1 is bounded for u > 0. Choosing 
|6|, |c| sufficiently small guarantees that ||<S^|| < e for any e > 0. Furthermore, 

(n n <g> p)^ n+1 (n n <g> p + w)- 1 = ([n n (/ n _! <g> c^n- 1 ] <g> g f ) (n n <g> p)(n n <g> p + u)- 1 . 

Since ||Q^|| = q can be made arbitrarily small by choosing |6|, |c| sufficiently small, the 
boundedness of (n n+ i + w) -1 will follow from a bound for the operator Il n (/ n _i ® c^II" 1 
which is uniform in n. Accordingly let us define for n > 1, 

A n = n n (/ n _i ^n- 1 . 

By imitating the derivation above, we obtain the recursion relation 

A n = (/ + ^ n _! ® Qt + 1 5 t) ( 7n _ 1 w ) + X _ An _ x gt + 1 5 t ) ~\ (A.l) 

It is immediate that ||Ai|| = ||u;|| < 1. We make the inductive assumption that 
||-4n-i|| < 1; then (A.l) implies the estimate 

11^.11 Sd + f + fMl-f-f)- 1 - (A.2) 

If we choose 

(1 + iu) 2 

then (A.2) implies that ||A n || < 1. Hence by choosing (A. 3) we obtain that A n is uniformly 
bounded for all n, and hence that (II n+ i + -u) _1 is bounded for all u > 0. Therefore the 
positivity of the density matrices is proved. 
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